Markov chain order estimation with conditional mutual information

Markov Chain Order estimation with Conditional Mutual Information

We introduce the Conditional Mutual Information (CMI) for the estimation of the Markov chain order. For a Markov chain of K symbols, we define CMI of order m, Ic(m), as the mutual information of two variables in the chain being m time steps apart, conditioning on the intermediate variables of the chain. We find approximate analytic significance limits based on the estimation bias of CMI and dev...

Markov chain order estimation with parametric significance tests of conditional mutual information

Besides the different approaches suggested in the literature, accurate estimation of the order of a Markov chain from a given symbol sequence is an open issue, especially when the order is moderately large. Here, parametric significance tests of conditional mutual information (CMI) of increasing order m, Ic(m), on a symbol sequence are conducted for increasing orders m in order to estimate the ...

Binary Feature Selection with Conditional Mutual Information

In a context of classi cation, we propose to use conditional mutual information to select a family of binary features which are individually discriminating and weakly dependent. We show that on a task of image classi cation, despite its simplicity, a naive Bayesian classi er based on features selected with this Conditional Mutual Information Maximization (CMIM) criterion performs as well as a c...

The Intrinsic Conditional Mutual Information

This paper is concerned with secret key agreement by public discussion: two parties Alice and Bob and an adversary Eve have access to independent realizations of random variables X, Y , and Z, respectively, with joint distribution P XY Z. The secret key rate S(X; Y jjZ) has been deened as the maximal rate at which Alice and Bob can generate a secret key by communication over an insecure, but au...

About the Mutual (Conditional) Information